FSR: A General Frequency-Oriented Framework to Accelerate Image Super-resolution Networks

Deep neural networks (DNNs) have witnessed remarkable achievement in image super-resolution (SR), and plenty of DNN-based SR models with elaborated network designs recently been proposed. However, existing methods usually require substantial computations by operating spatial domain. To address this issue, we propose a general frequency-oriented framework (FSR) to accelerate considering data characteristics frequency Our FSR mainly contains dual feature aggregation module (DFAM) extract informative features both transform domains, followed four-path SR-Module different capacities super-resolve the Specifically, DFAM further consists attention block (TABlock) context (SCBlock) global spectral information local information, respectively, while is parallel container that four to-be-accelerated branches. Furthermore, an adaptive weight strategy for trade-off between details recovery visual quality. Extensive experiments show our can save FLOPs almost 40% reducing inference time 50% other (e.g., FSRCNN, CARN, SRResNet RCAN). Code available at https://github.com/THU-Kingmin/FSR.